This invention relates to a system for enhancing the hearing ability of hearing impaired persons. More particularly, this invention pertains to the improvement of speech intelligibility for persons listening to equipment producing audio signals such as television receivers, recorded music, or radio units.
Hearing improvement aids have been under continuous development for many years. Recently, significant advances have resulted from the introduction of electronic components, electronic circuits and software developments. In the last few years, significant research has lead to a better understanding of the physiological and neurological mechanisms relating to the sense of hearing. Such research is directed to the causes of hearing impairments and possible solutions. Many types of hearing impairments can be treated with surgery or medication. For example, chronic ear infections, which can decrease hearing acuity, may be treated with antibiotics. Also, damaged eardrums can be repaired by surgery. Other ailments such as presbycusis (age related hearing loss) are ameliorated to a certain degree with hearing assistance equipment such as hearing aids.
Hearing impairment falls into four main categories: conduction loss, sensorineural loss, mixed loss, and central loss. Conduction loss is associated with problems in the outer and middle ear that prevent sounds from reaching the inner ear where they are converted from mechanical energy to electrical signals. Sensorineural loss involves either the inner ear or the auditory nerve. The inner ear contains thousands of sensory cells (haircells) that transform sounds into proper neural format to be transmitted to the brain via the auditory nerve. Problems with the sensory cells or auditory nerve exhibit the same results when hearing tests are performed. Mixed loss is a term used to represent a hearing impairment that involves both conduction and sensorineural loss. Central loss occurs when the hearing loss is not associated with conduction or sensorineural types of problems, but the brain itself has difficulty interpreting the signals received from the hearing process.
The invention presented here addresses three areas that represent significant problems for people who suffer from hearing impairments: background noise, room acoustics, and situations where the subject has lost virtually all of his or her hearing capabilities.
It is well known that background noise presents a problem for persons with normal hearing and even more severe problems to many people with impaired hearing. Background noise addressed by this invention falls into three categories. First, system or electrical circuitry background noise is inherent in all electrical equipment. Such system background noise has many sources including induction from ambient electromagnetic sources and non-linear circuitry introducing distortions into the desired electrical signal. Background system noise, if not mitigated, is mixed with the desired audio signals and is reproduced by the speaker system. A second type of background noise is the ambient noise created by machinery, other people, and other sounds that exist in the immediate environment of a person trying to discern spoken words. Ambient background noise has many sources such as crowded rooms (many people talking), air conditioners and fans, kitchen equipment, traffic and road noise, the hum of facsimile machines and computers, factory/industrial equipment, etc. A third type of background noise is defined as those components of an electronic audio signal that interfere with a hearing impaired persons ability to understand the speech component of the same signal. For example, a hearing impaired person watching a television program that has a person speaking and a siren in the background may have trouble resolving the speech. This interfering background noise differs from ambient background noise in that it is part of the sound being produced by the speaker system. Multiple speakers talking at the same time on an audio program presents such interfering background noise problems.
Several concepts and systems for the reduction of background noise exist. For instance, see U.S. Pat. Nos. 4,025,721; 4,461,025; 4,630,304; and 5,550,924, each of which is incorporated herein by reference.
The second environmental condition that causes hearing difficulties is related to room or environment acoustics. Techniques for improving audio quality for particular types of hearing impairments are of marginal value if audio speakers are in an environment having poor acoustics. Poor acoustical environment, whether in a private home, a car, a shopping mall, or sometimes even an auditorium, can make listening to a television, recorded materials, radio or a live performance, difficult even for a person with normal hearing. Sound waves emanating from speakers will contact every surface in the environment and the uncontrolled reflected, and to some extent the absorbed, sound waves will have an effect on the overall sound quality in the environment. The interaction of sound reflections with the incident sound waves can produce room resonance, resonance at natural frequencies, and standing waves. Research into minimizing these sound wave interference effects has resulted in speaker placement concepts and software techniques for acoustical design of enclosures, interior spaces and rooms in general. Signal processing techniques, wherein a digital audio signal is conditioned through software before being output to the speakers, have also been developed.
Even with the use of hearing improvement techniques such as environmental tuning to improve acoustics and control techniques to account for background noise, there still are situations where hearing impairment remains. For extreme cases of hearing loss, including total hearing loss, other methods have been developed. In one approach, the speech in an audio signal is isolated with sophisticated mathematical processing techniques. After the desired components of a particular audio signal are isolated, they can be analyzed and synthesized into textual equivalents of the original target speech sound. The speech, synthesized using a software program is then displayed as the text on a television screen or other display device. Speaker independent speech recognition is one technique to determine spoken words present any audio signal from a television, prerecorded playback device, live presentation, radio, or other source containing spoken words. Speech recognition algorithms process digital audio signals derived from an analog signal or inherently present in digital signals such as those used for digital television or audio broadcasts. Complicated signal processing algorithms, such as hidden Markov modeling (HMM), are implemented to resolve the speech in the presence of other speakers or other types of background noise. Once a speech signal is isolated it can be displayed as sub-titling or amplified to stand out from the other sounds in the audio signal.
Another sophisticated technique for the translation or conditioning of speech so that the actual speech can be textually or graphically presented is found in lip reading systems. The lip reading of a video signal incorporates established techniques used in computer vision. Mathematical or digital modeling of the face and lips of a speaker, singer, or the like, projecting words make computer vision lip reading a viable technique to translate or condition speech elements transmitted through a video signal.
Another element that is background to this invention is the evolution of expert systems. Expert systems are well known in the research community and are implemented in diverse systems today. An expert system is a problem solving technique and methodology that takes advantage of the knowledge base of experienced professionals and technicians who have many years of training and experience in a particular field. For example, in the medical field, expert systems use the knowledge of many experienced doctors to assist in the diagnoses of disease. Expert experience and knowledge is input into a cumulative database. The database can be searched by other doctors, technicians and interested parties to assist in the diagnoses of medical conditions based on particular patient symptoms. Expert systems use a forward or backward chaining process to answer posed questions. Facts input from a user become part of the database to be used in the chaining process. In a typical query, a doctor inputs the patient's current and/or past symptoms. Those symptoms are “facts” that aid the expert system in answering queries concerning the type of malady.
While systems and methods exist for improving hearing ability of the hearing impaired, for filtering background noise, and for compensating for room acoustics, a comprehensive integrated system and method using a combination of such technologies integrated with individual hearing loss profiles, modern computer vision, speech recognition, and expert systems all operated under the control of the hearing impaired individuals to improve speech intelligibility has not heretofore been described. Thus a need exists to provide such a comprehensive system and method to improve speech intelligibility for the hearing impaired.